Abstract

In the years to come, the oceanographic community faces a unique challenge related to ocean observatories. Even with the many legacy observatory systems operating today and with OOI observatories on the horizon, there is a large gap in the community between operational instruments and finished data products. Even if an instrument is part of an ocean observatory, there is no guarantee in today's mix of systems that the data will be available for processing and dissemination. MBARI has developed technology to help fill that gap both within the walls of MBARI and in the external community. Originally a component of the MOOS project, the Shore Side Data System has become an integral component of several MBARI core (and some non-MBARI) data streams and successfully bridges that gap between instrument and finished data products.

As it was envisioned during its development, the SSDS is now successful in managing all the metadata surrounding instruments, their deployments, and the data they produce. Furthermore, the SSDS tracks specific versions of software that produce data sets such that complete provenance of a data set can be provided. This type of system is unique within the oceanographic community and has served MBARI's operational data management needs well by allowing us to remove metadata assignments from data processing software resulting in more reusable and efficient code. Examples of this can be seen in the Mooring netCDF plot pages where a single module of software is used to process data from diverse multiple instruments into a common format. Before SSDS this kind of processing was done by multiple groups and individuals resulting in often incompatible data sets. One approximate measure of efficiency gained is that one programmer can now do the work of what used to take 3 or 4 people. It is currently responsible for managing the metadata and data for the following systems and their post processed data products:

1. MO Mooring (CIMT)
2. M1 Mooring
3. M2 Mooring
4. MSE Mooring (all four nodes)
5. AUVCTD
6. Bruce Howe's Aloha Mooring CTD and flourometer (ADCP will be soon).

In addition to its internal success at MBARI, the SSDS capabilities have been deemed desirable to the ORION OOI program and is, in fact, part of the OOI CyberInfrastructure proposal. Also, if MBARI wins the CGSN proposal, it is envisioned that they will need a system to "bridge" their development with that of the CyberInfrastructure. Because both the CI and the CGSN are being developed in parallel, it is likely that the CGSN will need some data management before the CI is available for that functionality. The SSDS could provide a system that the CGSN team could develop against the help them get started on the highest risk elements of the CGSN-CI interface.

In order to continue to support our internal data needs at MBARI and provide the most value to the external community, we are proposing more work on SSDS to add functionality to existing components as well as developing new pieces to complete the SSDS package. Project resources are being requested in 2008 for tasks that include:

• Improve metadata editing capabilities and client applications
• Provide database integrity checking tools
• Provide more useful and concise query and operational views of data producing systems
• Conduct maintenance on our existing and growing archive of data and metadata
• Allow SSDS to be distributed as an open source project

A detailed list of tasks is available on the project Wiki: http://oceana.shore.mbari.org:8081/display/SSDS/Tasks

Word document submitted 10 July 2007. 

Criteria 

This will be an infrastructure project. The criteria for infrastructure project evaluation is the following:

1. Importance: Does the project address an important problem in oceanographic research?
   1. There is a gap between ocean instrumentation and data management systems and applications
   2. SSDS (with SIAM) has been filling that gap
   3. Automatic capture of all metadata
   4. Capturing data provenance
2. Uniqueness: How unique is this contribution and well-suited for undertaking at MBARI?
   1. Not really an undertaking, but it operational at MBARI
   2. Due to its uniqueness it is being considered as a component in the ORION CI IO and external interests (Dalhousie, SOPAC)
3. Timeliness: Why should this project move forward now? What are the drivers?
   1. This is the year to break SSDS out of MOOS and have it stand on its own legs.
   2. It is clearly an operational component, while the future of other MOOS technology is not clear
   3. This is the year to include non-MOOS inputs/outputs to make it a easier to use institutional asset
4. Strategic Plan: Does the project demonstrate relevance to MBARI's strategic plan?
   1. Yes, particularly transfer of knowledge to external community.
   2. Particularly well position to help with OOI (both CI and CGSN if we win)
   3. Facilitates the response to opportunities to pass on data and understanding gained in pursuit of MBARI's research plan to organizations overseeing the environmental health of Monterey Bay and other locales
5. The Team: Is the team appropriate for the work, are they available, and are they committed?
   1. Team would consist mainly of Kevin Gomes and Mike McCann (with support from others as more data streams are integrated)
6. Prior Productivity: Has the project leadership been successful with prior support?
   1. SSDS has been successful to date and is the reason we are seeking to push SSDS outside of the MOOS envelope
7. Does the project demonstrate improvements in operation from year to year?
   1. Yes, this past year has seen large improvements in robustness and support for MSE development team. Many processes are moving to depend on SSDS (Mike's processing, Fred's OASIS - M0, M1, M2, NDBC Export, UW/Aloha mooring, WHOI used for MTM3 cable)
8. Does the effort have a significant impact on an important MBARI activity?
   1. Yes, currently supporting M0/CIMT, M1, M2, AUVCTD, UW, MARS/SENSORS Prototype, Could impact CGSN award and serve as bridge between CGSN development and CI development.
9. Does the project team periodically assess the needs or requirements of its beneficiaries?
   1. Definitely, we are constantly fielding requests from Engineering, Operations, Science, and the external community, but we are limited to respond by resources.
10. Will the effort benefit a large number of users?
    1. Operations: Better instrument management and operations status monitoring
    2. Science: More/Better interfaces to find and utilize data and associated processing and resources
    3. Support Engineering: Cut time to manage mooring data streams and data availability to outside community
    4. External community: get SSDS code base out there (this also cuts our time to fields requests from the community).

Salient points from the Strategic Plan:

1. Our capacity for understanding the complexity of the ocean, and for forecasting a realistic view of its future that we will partially create, is limited by the lack of technology for observing the ocean and maintaining a sustained presence in that harsh environment.
2. Goal: Transform and advance understanding of the most significant unsolved problems in oceanography by developing, adapting, and demonstrating innovative technologies.
3. Goal: Utilize those developments to discover and understand how the natural system operates, responds to, and interacts with anthropogenic influences.
4. Goal: Transfer the knowledge gained and the technology developed to communities outside of MBARI, including policy makers, government laboratories, resource managers, and the public.
5. MBARI technology is in demand for adoption by groups external to the institution, and that demand is met through external partnerships, licensing, copying, or other strategies as appropriate.
6. Look at: Natural rhythms of the complex ocean systems (Box 4), such as quantifying and understanding variability in the ocean food web on the seasonal, El Niño, and North Pacific Decadal Oscillation (PDO) time scales (emphasize time-series here).
7. Research Actions: Develop a data archive for Monterey Bay that can be easily accessed by users who are not data providers and which can be integrated seamlessly with related data sets from the larger oceanographic community.
8. Strategy B1: Participate in national initiatives that are aligned closely with MBARI's strategic plan and technology developments (Box 9), such as the National Science Foundation's Ocean Observing Initiative and National Oceanic and Atmospheric Administration's Ocean Exploration Program.
9. Strategy D2: Be alert for opportunities to pass on the data, models, and understanding gained in pursuit of MBARI's research plan to organizations overseeing the environmental health of Monterey Bay and other locales. (Box 11).

SSDS_DM_Improvements_2008.doc (application/msword)
SSDS_Hardening_2008.doc (application/msword)
SSDS_Hardening_2008.doc (application/msword)